feat(server): serve text reranking with AIPerf - #1687

Background

Closes #1669; part of epic #1667. Users need pip-installed AIPerf to benchmark real native relevance scores for one query and a fixed passage list. The native server already exposes text generation and embeddings; reranking needs its own native endpoint and qualification.

Exit Criteria

• NIM-shaped /v1/ranking calls the public native document-list Task, returning real finite scores sorted by relevance with original passage indices retained.
• Unsupported truncation, non-text inputs, streaming, and output filtering are rejected; request IDs, admission, terminal records, and native Task timing are preserved.
• The pinned text-only Nemotron pilot passes its unchanged family correctness contract and actual published-client qualification.

Implementation

• Add capability-based native reranking dispatch for the declared legacy reranking mode and compatible public document-relevance Tasks. Use the owning family's actual list operation; serving contains no model math, pooling, tokenization, or score calibration.
• Add strict NIM HTTP schemas, complete-score validation, stable ordering for ties, body/input limits, and capability metadata. Record passage counts and query/passage byte lengths alongside native Task timing.
• Reuse existing worker/admission/cancellation mechanics and qualification runner/request joins. Keep scores complete across the performance-bridge transport independently of diagnostic compaction.
• Add native, HTTP, transport, and rejection regressions; three fixed query/two-passage input examples; a three-step pip-client guide linked to existing build guidance; and a narrowly scoped pilot coverage matrix.

The public Task ABI, bundle format, family implementations, dependencies, and numerical criteria are unchanged. Update the private worker and control plane together.

Change categories

• Model or runtime behavior
• Public API
• ABI
• Bundle or artifact format
• Dependencies
• Documentation only
• CI or developer tooling

Validation

Commands and Results

Commands below substitute reproducible paths or environment variables for local environment paths. Full executed commands and generated results remain outside the commit.

PYTHONPATH=apps/server/python:apps/perf_serving:apps/aiperf_qual:core/builder:. \
  python -m pytest apps/server/python/tests apps/perf_serving/tests -q --import-mode=importlib
ctest --test-dir build --output-on-failure \
  -R '^server_(worker|sdk_worker|sdk_worker_process|embedding_worker|ranking_worker)$'
PYTHONPATH=core/builder:apps/benchmark:. python -m pytest \
  families/eagle_vlm/tests/test_e2e.py --e2e-model nemotron-rerank-vl-1b-v2 -q
PYTHONPATH=core/builder:apps/benchmark:. \
  python -m tools.community_ci source-quality --base github/main
npm --prefix website run test:model-support
npm --prefix website run build
git diff --check github/main...HEAD

• Native server/worker and eagle_vlm runtime build passed; 5 native serving tests passed.
• Current-head serving/performance-bridge Python regressions: 117 passed, 1 skipped (an optional reference dependency). Qualification suite with its declared plugin installed: 377 passed, 2 skipped. An initial local collection/run lacked AIPerf and plugin registration; installing the declared test plugin resolved the environment issue without source changes.
• Owning-family validation: 5 passed before the rebase, including the real checkpoint build/native/reference case. Revalidated the exact served bundle with the owning family's unchanged reference and ranking-agreement assertion after installing its declared dependency pin. The exact served bundle passed the same unchanged family reference and ranking-agreement assertion again on the current rebased head. No family oracle or threshold changed.
• Current-head source quality and architecture checks: 306 passed.
• Website inventory tests (14 passed) and Node 22 production build passed before the rebase. Current-head Stable and Dev documentation CI also pass.
• Published AIPerf 0.13.0 on the reconciled committed source completed 50/50 measured requests and 5/5 warmups, concurrency one. Every response returned the two finite real scores and original indices in the correct order. Zero client failures, unmatched request IDs, or missing metric exports. Three direct HTTP cases matched the same bundle's native document-list operation exactly.

Throughput uses the measured client request span, excluding warmups.

Hardware, Environment, and Revisions

Tested final head: fb3ed80718b8a7b8f76e2f431cea2eacc0032629; base: ad1d0e78232116cdd97687026c63f6e10353e92d (latest main).

One NVIDIA GB300; Linux aarch64; driver 580.105.08; CUDA toolkit 13.3; TensorRT 11.1.0.106; Python 3.12.3. Model nvidia/llama-nemotron-rerank-vl-1b-v2, checkpoint revision 9e95da054312436dfc703319dd2b793a3bee2465, FP16, context limit 685, one persistent replica. Dataset: the three checked-in query/two-passage records. AIPerf is the published PyPI package; no source checkout dependency. Build/load/startup/warmup are excluded from measured requests; client loopback latency and Task-call wall time remain separate.

Not Run / Remaining Gaps

Other rerankers, image/document multimodal inputs, Cohere/HF TEI formats, larger passage lists, and load/capacity sweeps are not qualified here. The public SDK dispatch path has source/build coverage; this hardware pilot uses the existing public reranking Task. Passage lists and passages/sec do not establish parallel GPU batching. The 50-request run is an integration example, not a statistical performance gate. Remote CI is reported separately.

Contributor Self-Review

• I have completed a self-review of this change.

Reviewed family ownership, declared Task selection, real score/index preservation, stable ties, strict unsupported inputs, timing boundaries, error/lane lifecycle, pip-client behavior, checkpoint identity, and evidence privacy.

Notes For Future Readers

This stage builds on the embedding serving mechanics delivered by PR #1684. It retains text and embedding endpoints and adds reranking without duplicating the shared operation/HTTP lifecycle. Both real pilots were rerun through the reconciled server: 50 measured requests and 5 warmups each, with full vectors/scores and no request-identity or metric gaps.

Start review at the ranking route/result checks, then native dispatch and transport. Compare document-list operations with document-list operations; separate single-document calls can use different execution shapes. The family retains the score contract. Generated bundles, benchmark reports, and environment state are excluded from this PR.

Risk level

• Low
• Medium
• High

The private dispatcher and admission path expand beyond text. Native/text regression tests, malformed-output checks, and real native/AIPerf qualification cover the affected behavior.

Final CI

All expected checks pass on fb3ed80718b8a7b8f76e2f431cea2eacc0032629, based on ad1d0e78232116cdd97687026c63f6e10353e92d: Stable Community CI, Dev Community CI, including GPU validation and cleanup, source quality, documentation, ownership, metadata, DCO, and automated review. The exact head has TRTMC Internal CI / Automated premerge gate = PASS, published through the source bridge. Both review findings are fixed and resolved. All three feature/fix commits were preserved unchanged by the final rebase; no numerical criteria or CI gates changed.